The invention broadly relates to data processing systems. The invention has particular application in a multitasking pen computer system in which a user enters hand printed characters and symbols using a stylus to be interpreted as text and gesture commands.
Pen computers are a class of computers that employ a stylus and sensor as an input device. A pen software subsystem, which may be integrated into or an adjunct to the operating system, processes pen input for the purposes of recognizing stylus input as mouse input commands, hand printed characters (text) or hand printed symbols known as gestures. Gestures are typically used to invoke a function such as opening or closing a window.
For example, the pen subsystem of the OS/2 operating system, Pen for OS/2, permits users to write characters directly into a graphical user interface (GUI) window which are subsequently sent to gesture and/or handwriting recognition engines for recognition. One default method of distinguishing between gestures and handwriting is used by the system by treating the hand printed shapes as gestures unless the window is one that accepts text, such as an edit field or a text entry window.
Preferably, the pen operating subsystem will facilitate pen input into all applications that are executed by the computer system, most of which are "unaware" of the stylus input. The typical application program today is written only to accept mouse and keyboard input and is unaware of the function rich programming interface allowed by pen input. To improve ease of use and provide good human factors, it is greatly preferred that the pen subsystem make these pen-unaware applications look and feel, from a user interface point of view, like the applications which are pen-aware and take advantage of the pen functionality.
Pen for OS/2 provides for the recognition of hand drawn shapes as either gesture symbols or as characters (text). Since the advent of pen-computing, designers have struggled to develop recognition engines that accurately recognize both gestures and text. Though tremendous improvements have been realized, even state of the art recognition engines still have problems distinguishing between gestures and text. This is due to the similar shapes of some gestures and some letters. Indeed, unless the system uses highly complex and user unfriendly gestures, because of the range of strokes and shapes in the alphabet and numerals, many of the useful gestures will have some similarity to existing letters.
As an example, any pen operating systems support the circle gesture because of its usefulness. However, if a circle is drawn on top of a word processor window, the system must decide if the user is: 1) drawing a circle gesture, 2) writing a zero, 3) writing an uppercase O, or 4) writing a lower case o? The same ambiguity applies to a single down stroke known as a `flick down` gesture. Is it a flick down gesture, the letter l, the letter I, or a 1 (one)? There are other examples, but these two suffice to demonstrate the difficulty of distinguishing gestures from text.
The ambiguity of the two examples above can be resolved in many cases by examining the context in which the character was written. In fact, a well written pen application will often resolve it by limiting the input of certain windows to numerals, alpha characters, alpha-numeric characters, or gestures only. Known as recognition constraining, limiting the possible results of the recognition engine significantly increases recognition accuracy. Another common technique of pen designers is to build a library of gesture shapes that are unlike any letter of the alphabet or digit. Unfortunately, this eliminates important gestures like circle, down flick, caret, down-left, X and question mark. Each of these gestures are intuitive, memorable and easily drawn and eliminating them would hurt usability. Pen For OS/2 users can write a question mark gesture on any window to bring up a help window. A problem exists when editing a memo; how could the user change a period to a question mark without having help pop up? In Pen for OS/2, any window that accepts keyboard input is enabled for both gesture and text recognition. Because the pen subsystem is largely unaware of the application environment, it is very difficult to distinguish gestures and text by context. Windows that do not accept text in pen aware applications do not have this problem because they are constrained to gesture only recognition.
Also worth noting is that Pen For OS/2 supports 26 letter gestures where the letter gesture is drawn by writing the upper case letter followed by a period or `tap`. The tap is the distinguishing feature between letter gestures and normal letters. However, in some cases, the tap introduces ambiguity: i.e. is it an upper/lower case letter and period?
One serious useability problem that persists is that users sometimes get different results after writing what seems to be the exact same character. This inability of the user to predict or control the recognition results leads to high levels of frustration and reduced productivity. Unfortunately, the prior art techniques that heavily constrain recognition and constrain recognition to a smaller subset of characters and symbols only are effective in certain situations, i.e. when properly designed into a pen aware application. For general, commonly used applications which are not pen aware, the user is hindered by the inability of the prior art to predictably distinguish between gestures and handwriting.
The present invention is an improvement over the prior art for distinguishing gesture input from handwriting input.